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E arly math ability is one of the best predictors of children’s math and 
reading skills into late elementary school. 1 Children with stronger 
math proficiency in elementary school, in turn, are more likely to 
graduate from high school and attend college. 2 However, early math 
skills have not historically been a major focus of instruction in preschool and 
kindergarten classrooms. 3 This brief presents the findings from a study of two 
early math programs — Making Pre-K Count and High 5s. 


The Making Pre-K Count and High 5s studies test two math programs to examine whether it is 
possible to improve children’s early math abilities, and whether improvements in this “linch¬ 
pin” outcome lead to impacts on children’s other short- and longer-term outcomes. The cur¬ 
rent analysis examines the cumulative effects of both programs on children’s math, language, 
and executive function skills in kindergarten. 4 The Making Pre-K Count program entailed 
a comprehensive redesign of both the content and teaching of math in the prekindergarten 
(pre-K) classroom, and the High 5s program provided a second year of math enrichment for a 
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1 Duncan et al. (2007). 

2 Duncan and Magnuson (2009). 

3 Ginsburg, Lee, and Boyd (2008); Lobman, Ryan, and McLaughlin (2005). 

4 Executive function refers to a set of skills that underlie children's self-regulation, potentially making 
it easier to attend to the learning tasks of school. 






BOX 1 

TWO EARLY MATH PROGRAMS: THE COMPONENTS 
OF MAKING PRE-K COUNT AND HIGH 5s 

Making Pre-K Count focused on children’s experiences in pre-K 
classrooms. It consisted of the following: 

• Building Blocks, a 30-week, evidence-based math curriculum 
designed by Drs. Doug Clements and Julie Sarama to take 
into account children’s natural developmental progression in 
math skills. Building Blocks includes four main components: 

1. Weekly whole-group math lessons 

2. Hands-on math materials for children to use in play 
centers 

3. Small-group math activities for teachers to conduct with 
three to four children at a time, allowingteachers more 
opportunity to work with children individually or in pairs 

4. Computer games that respond to the child’s skill level 

• Ongoing teacher training about the curriculum over two years 

• Coaching of teachers in math instruction two to four times 
a month over two years 

High 5s provided a supplemental math experience outside the 
classroom in kindergarten for some children who had Building 
Blocks in pre-K: 

• Small groups of three to four children with a facilitator trained 
in the program by Bank Street College of Education 

• 30-minute sessions, three times a week 

• Short, fun math activities designed to build on children’s 
pre-K math learning 


subgroup of children who received Making Pre-K 
Count in preschool. High 5s was designed to build 
on children’s pre-K experience using small-group 
math clubs — in which a trained facilitator works 
with three to four children on fun math activities 
three times a week outside the classroom — to 
supplement regular kindergarten instruction. Box 


1 provides more information about these two early 
math programs. 

These studies were designed as part of the Robin 
Hood Early Childhood Research Initiative, which 
was established to identify and rigorously test 
promising early childhood interventions. The initi¬ 
ative is a partnership between Robin Hood, one of 
New York City’s leading antipoverty organizations, 
and MDRC, a nonprofit, nonpartisan education 
and social policy research organization. Making 
Pre-K Count and High 5s, conducted in collabo¬ 
ration with Bank Street College of Education and 
RTI International, are also supported with lead 
funding from the Heising-Simons Foundation, the 
Overdeck Family Foundation, and the Richard W. 
Goldman Family Foundation. MDRC consulted 
with the Division of Early Childhood Education 
at the New York City Department of Education 
(DOE) and the Administration for Children’s 
Services’ Division of Child Care and Head Start 
throughout the study. (See Box 2 for information 
about an earlier report, which describes the short¬ 
term impacts of Making Pre-K Count on teacher 
practices and children’s outcomes in pre-K.) 

This brief provides a preliminary glimpse at the 
impacts on child outcomes in the kindergarten 
year and represents the cumulative effects of 
Making Pre-K Count in pre-K, and High 5s in 
kindergarten. The general pattern of findings is 
positive, with statistically significant impacts (or 
program effects that were in all likelihood not 
a result of chance) on three out of six measures 
assessing four child outcome domains: math, 
math attitudes, language, and executive function. 
Future reports will examine the separate effects of 
the two programs on children’s outcomes through 
third grade and will investigate how these results 
do or do not align with previous findings from this 
study and other studies of early math enrichment 
for children. 
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BOX 2 

MAKING PRE-K COUNT: PRE-K FINDINGS 
FROM 2016 

Previous findings to date from the Making Pre-K Count study, which are presented in more detail in 
the study’s pre-K report* include: 

• Implementation. The Building Blocks math curriculum was supported by strong coaching and training 
for teachers. Teachers generally implemented Building Blocks in their classrooms well, with good 
implementation of three out of the four main curricular components: whole group activities were 
conducted on 92 percent of the days that children were in attendance; hands-on math materials 
were available for children to play with on 93 percent of days; and teachers were able to cycle most 
children through a small group during 85 percent of the weeks when the curriculum was implemented. 
Teachers were able to get most children to the computer to play math-related games for 65 percent 
of the weeks, slightly less often than the other components. 

• Impacts on teachers’ instruction. Teachers taught an additional 12 minutes of math a morning, 
across more math domains, in program classrooms. This extra time was on top of an unexpectedly 
large amount of math instruction (35 minutes) already in place in New York City pre-K classrooms, 
possibly attributable to the rollout of a number of early childhood initiatives at the time of the 
study, including the New York State Common Core pre-K standards for math and literacy and full- 
day pre-K for all four-year-olds in the city. The program also had small positive impacts (effect size 
= 0.45) on the quality of math instruction, although this finding did not generalize to the quality of 
all instruction in the classroom! 

• Impacts on children’s outcomes in pre-K. Despite the effects on teacher practice, there were no 
impacts on children’s math, language, or self-regulation outcomes at the end of pre-K. There was, 
however, an impact on children’s math skills in the fall of the pre-K year (effect size = 0.31); this 
effect seemed to have faded out by the end of pre-K, however. 

These findings do not align with previous published studies about the Building Blocks curriculum, 
which find moderate to large impacts on children’s math skills at the end of pre-K. These early findings 
presented a number of questions about what might explain the divergence from other studies’ findings 
and about likely effects in future years. 


* Morris, Mattera, and Maier (2016). 

f An effect size is a statistical measure of the magnitude of an impact that is standardized. (That is, it has 
the same meaning no matter what unit is used to measure the impact.) 
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STUDY DESIGN 


Making Pre-K Count took place in 69 pre-K sites 
that comprised 173 classrooms serving over 2,700 
mostly low-income children of color in New York 
City. 5 Thirty-five sites were randomly assigned to 
receive Making Pre-K Count (the program group), 
which entailed two years of an evidence-based, 
developmental^ appropriate math curriculum 
called Building Blocks, along with teacher training 
and in-classroom coaching. The remaining 34 sites 
were randomly assigned to continue their typi¬ 
cal pre-K programming (“pre-K as usual,” or the 
control group) and did not receive Making Pre-K 
Count or High 5s. 

As children in the Making Pre-K Count program 
group sites entered kindergarten, a randomly 
selected subset was assigned to also receive a sec¬ 
ond year of math through High 5s, which provided 
small-group math club instruction in kindergar¬ 
ten. As a result, approximately one-fourth of the 
children included in the Making Pre-K Count 
program group in this analysis also participated 
in the High 5s clubs, while the remainder did not. 
Further, none of the children in the control sites 
participated in the High 5s clubs in kindergarten. 
As such, the High 5s study examines whether a 
supplemental dose of math in the kindergarten 
year, which was designed to reinforce and build on 
the new approach to math that preschoolers expe¬ 
rienced with Making Pre-K Count, further helps 
improve children’s math skills. 

The children were followed into kindergarten to 
assess the longer-term impacts of Making Pre-K 
Count and the short-term effects of High 5s. 


5 The analytic sample includes a randomly selected 
subset of 1,382 children who were chosen to 
participate in data collection. 


FINDINGS 

The current analysis is a preliminary look at the 
impacts on children’s outcomes in kindergar¬ 
ten across the Making Pre-K Count and High 5s 
initiatives. The findings described in this brief do 
not attempt to disentangle these two samples or 
studies. Therefore, any impacts in these analyses 
may be driven by Making Pre-K Count in pre-K, 
High 5 s in kindergarten, or the combination of the 
two. Future analytic work, which will be presented 
in later reports, will investigate the individual 
impacts of each program. The current analysis 
examines the cumulative effects of both programs 
on children’s math, language, and executive func¬ 
tion skills in kindergarten. 

How Were Children’s Skills Measured 
in Kindergarten? 

A number of individually administered direct 
assessments were conducted with children at 
the end of the kindergarten year. Children’s 
math skills were measured using two assessment 
instruments, both of which involve one-on-one 
engagement on a series of math activities between 
the child and an assessor. A detailed measure, 
Research-Based Early Math Assessment-Kinder¬ 
garten (REMA-K), was used to assess children’s 
knowledge of numbers, operations, geometry, pat¬ 
terning, and measurement. 6 A more global meas¬ 
ure (Woodcock-Johnson Applied Problems) was 
used to examine children’s quantitative reasoning. 7 
Children’s attitudes toward math were assessed by 
asking them to point to a set of five faces, ranging 
from smiling to sad. 


6 REMA-K is an adaptation of the REMA (Clements, 
Sarama, and Liu, 2008). It consists of a subset 
of items that are most appropriate for this age 
range and reflect the following key math topics: 
number/operations, measurement, patterning, and 
geometry. 

7 Woodcock, McGrew, and Mather (2001). 
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Because of the language-rich nature of the inter¬ 
ventions, which asked children to explain their 
mathematical thinking as they solved problems, 
children’s language skills were assessed using a 
measure of their receptive vocabulary (ROWPVT-4) 

— that is, their understanding of spoken words. 8 
Further, because math requires children to prob¬ 
lem solve and shift between different math con¬ 
cepts, two measures of executive function were 
also assessed. Working memory, or a child’s ability 
to retain pieces of information and move them 
around mentally, was assessed by asking children 
to point to a set of ordered blocks in the reverse 
order to which they were presented (Corsi Blocks 
backwards); and inhibitory control, or how chil¬ 
dren stop themselves from providing an expected 
response, was assessed using a computer game 
that asked children to alternate between touch¬ 
ing the same or the opposite side of a tablet when 
presented with different pictures and rules (Hearts 
and Flowers). 9 

What Did the Study Find? 

Table r shows comparisons between children in the 
control group and children in the program group. 10 * 
All of the children in the program group received 
enhanced math instruction in pre-K through Mak¬ 
ing Pre-K Count, and one-fourth received addi¬ 
tional math instruction in kindergarten through 


8 Martin and Brownell (2011). 

9 Corsi (1972); Wright and Diamond (2014). 

10 Multilevel modeling accounted for the clustering of 
children within their original pre-K sites. Program 
impacts were estimated by comparing mean 
outcomes in kindergarten for the group assigned 

to the program during pre-K with corresponding 
means for children assigned to the control group 
during pre-K, with an adjustment for a small set of 
background characteristics and dummy variables 

for random assignment blocks. As mentioned 
earlier, one-fourth of the group assigned to the 
Making Pre-K Count program also received the High 
5s intervention. 


High 5s clubs. Positive impacts were observed in 
three out of the six measures examined across four 
domains of children’s outcomes assessed in kin¬ 
dergarten. The findings show positive impacts on 
one measure of children’s math skills, a measure of 
children’s attitudes toward math, and one measure 
of executive function. 

The first row of the table shows that the inter¬ 
ventions appear to have a statistically significant, 
modest impact on a detailed and comprehensive 
assessment of children’s math skills (REMA-K). 
This effect is similar to the size of effects seen in 
other large-scale evaluations of preschool curric¬ 
ula one year after the children leave preschool. 11 
(For instance, a previous large study of Building 
Blocks in three cities found impacts with effect 
sizes ranging from 0.13 to 0.19 in the kindergarten 
year, although those impacts were not statisti¬ 
cally significant.) 12 On the more global measure of 
children’s math skills (Woodcock-Johnson Applied 
Problems), shown in the second row of the table, 
children in both groups scored similarly. This 
global math measure focuses more on basic count¬ 
ing and less on geometry — a unique focus of the 
math interventions evaluated here — and may be 
less sensitive to the interventions’ effects as a result. 
Children who received enriched math instruction 
also reported a slightly more positive attitude 
toward math than did children who did not receive 
these programs, although both groups reported 
generally positive feelings on this question (with 
an average of around 3.5 on a scale of 1 to 5). 

At the outset of this study, it was hoped that 
greater instruction in math might improve aspects 


11 See, for example, Bierman et al. (2014); Morris et 
al. (2014); Sarama, Clements, Wolfe, and Spitler 
( 2012 ). 

12 Hofer, Lipsey, Dong, and Farran (2013). Effect size 
is a measure of the magnitude of an impact that is 
standardized; that is, it has the same meaning no 
matter what unit is used to measure the impact. 
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TABLE 1 

CHILD-LEVEL IMPACTS OF ENRICHED MATH INSTRUCTION, 
SPRING OF THE KINDERGARTEN YEAR 


OUTCOME 

PROGRAM 

GROUP 

MEAN 

CONTROL 

GROUP 

MEAN 

DIFFERENCE 

(IMPACT) 

STANDARD 

ERROR 

EFFECT 

SIZE 3 

Math 

Detailed math score (REMA-K) b 

38.81 

37.68 

1.13** 

0.48 

0.13 

Global math score (Woodcock-Johnson Applied Problems) 0 

104.37 

104.37 

0.00 

0.72 

0.00 

Math attitudes 

Children’s attitudes toward math d (1-5) 

3.59 

3.44 

0.14* 

0.08 

0.09 

Language 

Receptive vocabulary 6 

98.32 

97.61 

0.71 

1.02 

0.05 

Executive function 

Inhibitory control' (0-1) 

0.69 

0.68 

0.01 

0.01 

0.04 

Working memory® 

2.41 

2.22 

0.19** 

0.08 

0.13 

Sample size 






Blocks 

16 

16 




Sites 

35 

34 




Children 

698 

684 





SOURCE: MDRC calculations based on the direct child assessments administered in spring 2016. 

NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

a Effect size is calculated by dividing the impact of the program (the difference between the means for the program group and the 
control group) by the standard deviation for the control group. 

b This study used an adaptation of the Research-Based Early Math Assessment-Kindergarten (REMA-K; Clements, Sarama, and Liu, 
2008). Item selection represents the full range of early mathematics competencies applicable within the prekindergarten, kindergar¬ 
ten, and early first grade years. The score in this table is the Item Response Theory (IRT)-based score. 

'Woodcock-Johnson Applied Problems is a subscale of the Woodcock-Johnson III Tests of Achievement (Woodcock, McGrew, and 
Mather, 2001). The score is age normalized to 100, with a standard deviation of 15. 

d The research team at MDRC created an assessment to measure children's attitudes toward math and school. Children were asked 
to use a showcard that displayed a range of five sad to smiling faces to describe how happy or unhappy school and math made them 
feel. A 1 rating indicates that they felt very unhappy and a 5 rating indicates that they felt very happy. 

deceptive One-Word Picture Vocabulary Test (ROWPVT-4; Martin and Brownell, 2011). The score is age normalized to 100, with a 
standard deviation of 15. 

f Hearts and Flowers (Wright and Diamond, 2014) is a computerized task that measures inhibitory control. The proportion correct 
score assesses how many trials a child gets correct out of 33 “mixed” trials where children must select the button on the same side 
if a heart appears and on the opposite side if a flower appears, excluding trials with response times faster than 200 milliseconds. 

g Corsi Blocks (Corsi, 1972; Lezak, 1983). A child is asked to repeat a sequence of blocks tapped by an assessor, tapping the blocks 
in reverse order. The child begins with a sequence of two blocks and more blocks are added to the sequence. Children receive a score 
of zero if they fail the first two trials; otherwise, the score reports the highest number of blocks the child is able to tap in correct order 
in two attempts. 
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of young children’s executive function skills. In 
fact, children in the program group performed 
better on a task of working memory but not on a 
task of inhibitory skills in kindergarten than chil¬ 
dren who received a typical math experience. 

WHAT’S NEXT? 

These preliminary findings demonstrate that an 
enhanced early math experience can have pos¬ 
itive, albeit modest, impacts on children’s math 
and executive function skills in kindergarten. 

This conclusion seems to align with the pattern of 
findings from a Building Blocks study that took 
place in San Diego, a context similar to New York 
City in that it has a relatively high level of math 
instruction in pre-K and a large Hispanic popu¬ 
lation. 13 In that study, there were no statistically 
significant impacts at the end of pre-K, but pro¬ 
gram effects similar in magnitude to those in this 
report emerged by the end of kindergarten. The 
magnitude of the impacts described here is similar 
to the size of impacts one year after implemen¬ 
tation in other studies of preschool curricula, 
which typically have ranged from around o.io to 
0.30. 14 However, few studies have followed up with 
children past elementary school, making it diffi¬ 
cult to assess the long-term implications of these 
findings for children’s outcomes into adolescence 
and adulthood. 

Future analysis of the Making Pre-K Count and 
High 5s programs will be critical for interpreting 
the preliminary findings and for more completely 
assessing preschool and kindergarten program 
impacts: 


• First, as discussed earlier, it is not clear whether 
the effects described in this brief occur directly 
though the Making Pre-K Count math program, 
the High 5s kindergarten math clubs, or the com¬ 
bination of the two programs. Future analyses will 
examine the impacts of High 5s alone on children’s 
kindergarten outcomes and will attempt to dis¬ 
entangle the two programs’ effects to identify the 
unique impact of Making Pre-K Count alone. 

• Second, further analysis will examine the role of 
measurement in the pattern of results and why 
math gains are evident in a detailed measure of 
children’s math competencies but not another 
more global, nationally normed measure of chil¬ 
dren’s math skills. Additional analysis will aim to 
determine whether different measures were more 
or less sensitive to impacts in particular areas of 
math learning, (numeracy, patterning, or geome¬ 
try), the latter of which was a particular focus of 
these math programs. 

• Third, future analytic work will delve into sub¬ 
group analyses to try to explain the pattern of 
impacts more deeply and to examine whether 
particular groups of children benefit more or less 
from the math interventions. 

• Finally, further follow-up of children who were in 
the Making Pre-K Count and High 5s programs 
is critical for understanding whether these gains 
persist as children advance through successive 
elementary school years. 


13 Clements et al. (2016). 

14 See, for example, Bierman et al. (2014); Morris 
et al. (2014); and Sarama, Clements, Wolfe, and 
Spitler (2012). 
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